The town was organized from parts of the towns of Hillsdale, Chatham, and Canaan, March 28, 1818. A little more than one-fifteenth of the present town was taken from Chatham, a little over one-eighth from Canaan, and a little less than five-sixths from Hillsdale. From the fact that among the first settlers there were no less than twelve families of Spencers, the north part of Hillsdale had been known from the first as "Spencer's-town." This name finally attached itself simply to the village, and when the division of the town was being talked up it was proposed to call the new town "New Ulm." When the bill erecting it passed the Legislature, however, Martin Van Buren, then a State Senator, and who, being an ardent admirer of the great Napoleon, was somewhat incensed at one of his political opponents (Elisha Williams, if we mistake not), who had succeeded in having a town in Seneca county christened "Waterloo," leaped to his feet and moved to amend by calling the new town "Austerlitz." Having carried his point, he retired to his seat, saying "There's an Austerlitz for your Waterloo."